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Commander’s  Mission  Statement 


I'd  like  to  introduce  you  to  the  Air  Force 
Human  Resources  Laboratory  (AFHRL).  The 
basic  purpose  of  this  Laboratory  is  to  conduct 
research  and  development  in  logistics,  training, 
and  personnel  technologies.  Our  research 
projects  include  programs  in  personnel  selection, 
classification  and  management.  They  cover 
programs  in  technical  education  and  training, 
flying  training,  and  team  training.  We  conduct 
R&D  to  develop  simulators  for  maintenance  and 
flight  training.  And,  finally,  the  Laboratory  carries 
out  research  in  the  logistics  and  human  factors 
areas  of  weapon  system  acquisition  and  combat 
maintenance. 

Since  the  largest  single  item  in  the  Depart¬ 
ment  of  Defense  budget  is  the  cost  of  personnel, 
and  their  training  and  administrative  support, 
there  is  a  greater  possibility  for  cost  savings 
from  the  human  resources  technology  area  than 
from  all  other  technology  areas  combined. 
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FRONT-END  ANALYSIS  GUIDELINES 


I.  INTRODUCTION 


A.  Guideline  Objectives 

The  guidelines  that  follow  are  designed  to  assist  Air  Force  Human  Resources  Laboratory 
(AFHRL)  personnel  in  conceptualizing,  planning,  and  conducting  front-end  analysis  (FEA)  studies. 
They  are  suggestive,  rather  than  mandatory.  There  is  no  single  model  for  an  FEA,  since  each 
FEA  undertaken  should  reflect  the  nature  of  the  decision  to  be  made.  Those  conducting  an 
FEA  are  expected  to  use  techniques  and  procedures  that  meet  the  objectives  of  the  analysis 
being  conducted,  given  the  state  of  knowledge  about  research  and  development  (R&D)  products, 
baseline  conditions,  and  systems  alternatives  that  exist  at  that  time. 


B.  Front-End  Analysis 

People  with  differing  backgrounds  use  the  term  front-end  analysis  in  different  waws.  Many 
people,  for  example,  use  the  term  FEA  to  specify  the  studies  conducted  to  spec.iy  ir  human 
factors  requirements  of  new  weapon  systems. 

At  AFHRL,  the  term  "FEA"  is  used  more  generally  to  describe  tne  analytic  studies  that  are 
conducted-  before  as  well  as  during  the  R&D  process--to  help  plan  a  variety  of  R&D  activities. 
It  encompasses  many  different  kinds  of  analysis,  possibly  including:  forecasts  of  future 
requirements,  assessments  of  emerging  technologies,  research  and  development  (R&D)  plans 
evaluation,  definition  and  comparison  of  alternative  system  configurations  and  deployment 
strategies,  product  value  estimation,  benefit  estimation,  life-cycle  cost  (LCC)  analysis,  and 
advanced  planning  studies. 

FEAs  at  AFHRL  are  designed  to  save  resources  by  helping  R&D  managers  evaluate  a  wide 
variety  of  important  planning  alternatives  before  the  R&D  investment  gets  too  big.  They  are 
not  economic  analyses  (EAs),  although  many  of  the  methods  and  guidelines  for  EAs  may  apply 
(see  section  1C). 

It  is  conceivable  that  an  FEA  could  be  conducted  by  making  a  half  dozen  telephone  calls 
to  requirement  managers  and  experts  In  a  particular  field.  The  person  who  conducts  the 
telephone  calls  may  be  satisfied  with  a  quick  answer  to  some  nagging  question.  Other  FEAs 
may  require  hundreds  of  thousands  n  contract  dollars  and  1  or  2  years  to  conduct.  Regardless 
of  the  time  required,  if  the  decision  is  an  important  one,  it  is  usually  a  good  idea  to  document 
the  analysis  in  writing.  This  permits  ofher  people  to  understand  why  you  decided  to  do  things 
one  way  rather  than  another,  and  can  be  very  important  if  someone  else  has  to  take  over  the 
R&D  effort  for  which  the  FEA  was  conducted. 


C.  Economic  Analysis 


Economic  analysis  (EA)  is  a  systematic  approach  to  the  problem  of  choosing  how  to  use 
scarce  resources  It  reveals  the  present  value  of  the  monetary  costs  and  benefits  associated 
with  all  alternatives  under  consideration,  and  provides  an  accurate  and  complete  as  possible  a 
picture  of  nonmonetary  costs  and  benefits.  For  guidance  regarding  how  to  conduct  economic 
analysis  studies  see  Air  Force  Regulation  (AFR)  173-15,  Economic  Analysis  and  Program  Evaluation 
for  Resource  Management. 
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In  the  Air  Force,  EAs  are  not  required  {paragraph  l-l-d(4),  AFR  173-15)  if  the  costs  of 
conducting  the  analyses  outweigh  the  potential  benefits  to  decision  makers.  EAs  are  consequently 
not  conducted  for  6.2  and  6.3  R&D,  since  the  nature  of  the  alternatives  for  R&D  efforts  is  not 
specific  enough  to  justify  detailed  cost-analytic  studies  of  this  type. 

Other  uses  of  AFHRL  funds  (e.g.  computer  equipment,  lease  versus  purchase  decisions  for 
large  R&D  support  facilities)  are  not  covered  by  this  exemption,  and  EAs  for  such  purposes 
are  required  If  there  is  a  lasting  resource  commitment  that  Involves  more  than  $1,000,000  or 
if  the  annual  recurring  costs  are  expected  to  exceed  $100,000. 

The  Human  Systems  Division  Comptroller  (HSD/AC)  will  provide  guidance  and  assistance 
regarding  the  need  for  EA  studies,  and  will  certify  them  upon  completion.  As  HSD  OFR  (Office 
of  Primary  Responsibility)  for  EAs,  HSD/AC  is  required  to  do  these  sufficiency  reviews  and  must 
formally  certify  EAs  that  are  to  be  briefed  or  forwarded  outside  of  HSD. 


D.  FEA  Policies  at  AFHRL 


The  conditions  under  which  AFHRL  personnel  conduct  FEAs  are  specified  in  AFHRLR  173-1, 
Front-End  Analysis  (FEA)  Studies.  Generally  speaking,  AFHRL  conducts  FEAs  for  the  period 
covered  by  its  6-year  research  and  technology  plan.  An  FEA  (AFHRLR  173-1)  is  required  for 
all  6.3  efforts.  FEAs  may  also  be  conducted  for  6.2  efforts  that  involve  the  development  of 
products  that  are  transitioned  directly  to  customers. 

There  is  no  requirement  to  conduct  an  FEA  for  every  work  unit.  Many  work  units  are  a 
continuation  of  work  that  has  been  in  process,  and  do  not  need  an  FEA.  Other  work  units 
may  be  structured  in  such  a  way  that  the  needed  FEA  is  an  intrinsic  part  of  the  R&D  itself 

In  R&D,  one  step  leads  to  another.  It  is  consequently  not  unusual  for  the  first  FEA  to  lead 
to  another  step  in  the  FEA  process.  For  example,  an  FEA  concerned  with  future  requirements 
may  confirm  that  work  is  needed.  A  second  FEA  concerned  with  the  assessment  of  emerging 
technologies  may  be  needed  to  confirm  that  additional  AFHRL  R&D  is  needed;  and -assuming 
that  additional  R&D  really  Is  needed-yet  another  FEA  may  be  required  in  order  to  evaluate 
system  configurations  and  deployment  strategies  before  a  large  scale  development  effort  is 
initiated. 


E.  Reviews  of  Work  Unit  Proposals  and  Plana 

The  Special  Projects  Office  (AFHRL/SA)  is  responsible  for  special  projects,  plans,  and 
front-end  analysis  studies.  As  part  of  the  group's  general  review  responsibilities  for  FEAs,  plans 
analysts  in  SA  review  work  unit  proposals  for  6.3  and  6.2  efforts  from  the  viewpoint  of  whether 
FEAs  are  necessary,  and  make  recommendations  to  divisions.  The  analysts  also  review  work 
unit  plans,  task  orders,  and  statements  of  work  (SOW)  to  determine  whether  the  FEAs  recommended 
before  work  unit  approval  either  have  been  done  or  are  scheduled  to  be  done. 

On  request,  SA  analysts  will  assist  division  and  staff  offices  by  reviewing  plans  for  FEAs 
and  consulting  with  personnel  regarding  improved  methods  and  procedures. 
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F.  SA-Sponsored  FEAs 


Assuming  that  the  needed  resources  are  available,  FEAs  by  SA  analysts  will  be  authorized 
by  the  Director,  Special  Projects  Office  (SA),  under  one  or  more  of  the  following  conditions: 

1  The  FEA  deals  with  new  mission  areas  or  new  technological  developments  that  are  not 
yet  part  of  any  division's  program; 

2.  The  FEA  deals  with  alternatives  that  do  not  fall  entirely  within  one  division’s  mission 
area; 

3.  A  division  does  not  have  the  expertise  or  resources  to  carry  out  a  needed  FEA  and 
requests  assistance;  or 

4  The  FEA  is  directed  by  the  Commander. 


G.  Questions  Answered  by  FEAs.  Typical  questions  answered  by  FEAs  are 

Do  future  Air  Force  requirements  really  require  R&D,  or  can  the  problem  'olved  in 
some  other  fashion? 

In  view  of  the  work  going  on  in  industry  and  universities,  does  the  Air  Force  really  need 
to  conduct  R&D  of  its  own? 

What  systems  alternatives  should  be  investigated? 

What  is  the  best  plan  for  conducting  the  R&D? 

Which  option  looks  best  from  a  life-cycle-cost  point  of  view? 

Which  options  provide  the  most  benefits  for  the  Air  Force? 

What  type  of  payoff  will  the  Air  Force  get  for  its  investment  in  the  proposed  system? 
Can  a  "crash"  program  be  justified  by  the  benefits  that  would  accrue? 

How  can  the  risks  inherent  in  the  R&D  be  minimized? 

What  is  the  impact  of  minor  fluctuations  in  external  variables  and  conditions  on  the  value 
of  the  systems  to  be  developei  ° 

II.  PLANNING  FEAS 


A.  Pre-Planning  Surveys 

At  AFHRL,  R&D  In  any  area  is  preceded  or  accompanied  by  a  literature  search,  and  FEAs 
are  no  exception.  When  an  FEA  is  planned,  it  is  important  that  this  literature  search  include 
FEAs  that  may  have  been  conducted  by  other  organizations.  Special  attention  should  be  given 
to  the  possibility  that  relevant  FEA  work  may  already  have  been  conducted  by  the  Development 
Planning  Office  in  HSD  headquarters.  HSD/XR  (Deputy,  Development  Planning  Office)  publishes 


two  types  of  special  area  planning  documents  for  HSD  use:  Requirement  Identification  and 
Technology  Assessment  Summaries  (RIATAS),  and  Special  Emphasis  Area  Plans  (SEAPs).  The 
SEAPs  are  updated  annually. 


B.  Decisions  About  Which  Type  of  FEA  to  Conduct 

Exhibit  il-1  is  a  list  of  questions  that  should  be  asked  before  FEAs  are  conducted.  As 
indicated  in  the  note  at  the  top  of  the  page,  the  more  "yes'1  answers  to  the  questions  in  each 
set,  the  more  logical  it  is  to  consider  a  study  of  the  type  indicated  in  the  heading  for  that  set 


C.  The  Sequential  Logic  of  FEAs 

There  is  a  definite  sequential  logic  or  priority  order  when  decisions  are  made  about  FEAs 
The  first  step  is  to  fully  understand  future  requirements  (FEA  type  A)  and  the  technology  that 
is  available  to  deal  with  these  user  requirements  (FEA  type  B).  It  is  also  desirable  to  study 
systems  change  alternatives  (FEA  type  C)  before  the  R&D  plans  are  prepared.  Since  budgets 
and  time  schedules  are  important  considerations  in  decisions  to  conduct  R&D,  most  analysts 
want  to  prepare  a  strawman  set  of  R&D  plans  and  evaluate  R&D  planning  alternatives  (FEA 
type  D)  before  they  decide  to  go  further. 

Once  the  base  case  and  the  alternatives  have  been  defined,  and  it  has  been  established 
that  the  R&D  plans  are  both  feasible  and  fundable,  the  analyst  may  want  to  explore  the  cost 
effectiveness  of  systems  change  alternatives.  If  the  R&D  product  is  tangible  and  it  is  easy  to 
assign  dollar  values,  the  analyst  may  elect  to  conduct  a  product  value  estimation  study  (FEA 
type  E).  However,  most  R&D  products  at  AFHRL  will  not  be  this  tangible. 

Product-value  comparisons  can  be  greatly  simplified  by  setting  up  equal-benefit  and  equal-cost 
alternatives.  These  alternatives  permit  the  analyst  either  to  use  benefit  estimates  to  compare 
equal-cost  alternatives  (FEA  type  F)  or  to  use  cost  estimates  to  compare  equal-benefit  alternatives 
(FEA  type  G).  Both  types  of  information  are  required  if  the  costs  or  benefits  of  the  options 
under  consideration  cannot  be  defined  in  such  a  way  that  they  are  equivalent -but  it  is  better 
to  avoid  unequal-cost,  unequal-benefit  studies  If  possible. 

Advanced  planning  studies  (FEA  type  H)  may  also  be  conducted.  Most  studies  of  this  type 
require  a  basic  network  planning  diagram  with  scheduled  milestones,  product  value  estimates, 
product  delivery  dates,  and  product  utilization  periods.  Predicted  variations  in  the  product  value 
estimates  and  utilization  periods  are  used  to  help  evaluate  such  things  as  R&D  scheduling 
alternatives  (e  g.  "crash"  programs),  the  consequences  of  the  risks  taken  at  critical  points  (risk 
analysis),  and  the  sensitivity  of  the  R&D  program  to  minor  fluctuations  in  the  values  of  key 
parameters  and  situational  constraints  (sensitivity  analysis). 
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Exhibit  ll-l .  Questions  to  Help  Decide  Which  Type 
of  FEA  to  Conduct 

NOTE:  All  questions  should  be  answered,  since  It  is  possible  that  more  than  one  type 
of  FEA  is  needed  Those  FEAs  listed  early  in  the  list  (A-D)  should  be  considered  during 
the  early  R&D  planning  stages,  regardless  of  whether  a  Program  Objective  Memorandum 
(POM)  or  Program  Management  Directive  (PMD)  already  exists.  Those  listed  later  (E-H) 
are  usually  deferred  until  after  the  nature  of  the  deliverables  Is  better  known.  The  more 
yes'1  answers  to  the  questions  in  each  set,  the  more  reason  to  believe  that  an  FEA  of 
that  type  should  be  conducted. 


A.  FEA  to  forecast  future  requirements1 

1  Is  the  Air  Force  developing  a  new  system  that  has  not  been  studied  before? 

2.  Are  the  user’s  problems  poorly  defined? 

3.  Are  the  conditions  under  which  the  R&D  would  be  applied  poorly  understood? 

4.  Is  there  a  need  to  explore  additional  systems  alternatives? 

5.  Is  it  possible  that  some  other  alternative  (including  man^emeiit  decisions  not 
requiring  R&D  by  AFHRL)  will  eventually  meet  or  char  je  the  user’s  requirement? 

6  Is  additional  information  needed  about  the  long  range  consequences  of  new  tactics 
or  weapon  systems  design  alternatives? 

B  FEA  to  assess  emerging  technologies1 

1  Is  the  current  state  of  the  art  inadequate? 

2  Are  Air  Force  applications  of  new  technologies  uncertain? 

3  Are  the  contributions  to  be  expected  from  parallel  efforts  poorly  understood? 

4  Does  more  consideration  need  to  be  given  to  the  impact  of  important  technological 
developments  in  other  fields? 

5  Are  important  changes  expected  in  the  state  of  the  art  due  to  recent  developments 
in  closely  related  fields  (  g.  computer  technology)? 

C  FEA  to  evaluate  systems  alternatives 

1.  Do  systems  alternatives  need  clarification? 

2  Are  tradeoff  studies  needed  to  help  evaluate  systems  deployment  alternatives? 


'  Relevant  HSD/XR  documents  should  be  checked  carefully  before  conducting  FEAs  of  these  two  types.  FEAs  to  forecast 
future  requirements  and  assess  emerging  technologies  are  regularly  conducted  by  HSD/XR. 
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3.  Will  the  proposed  cost-effectiveness  estimates  help  to  screen  out  or  clarify  systems 
design  alternatives? 

4.  Are  cost-benefit  comparisons  needed  to  justify  resource  allocation  decisions? 

5.  Is  the  proposed  effort  large  enough  and  important  enough  to  justify  detailed 
effectiveness  studies  of  systems  alternatives? 


D.  FEA  to  evaluate  R&D  planning  alternatives 

1.  Are  detailed  R&D  plans  needed  to  help  define,  sell,  or  prepare  budgets  for  future 
plans? 

2.  Are  time  and  resource  requirements  uncertain? 

3.  Is  an  R&D  budget  path  or  "baseline"  cost  estimate  needed  to  plan  for,  initiate,  or 
continue  work? 

4.  Are  precise  contract  cost  estimates  needed? 

f.  Do  the  costs  and  benefits  of  important  R&D  planning  alternatives  need  to  be 
.ompared? 

6.  Is  detailed  information  about  a  basic  R&D  plan  needed  to  conduct  advanced  planning 
studies? 


E.  Product  value  estimation  FEAs2 

1.  Is  it  necessary  to  compare  the  costs  of  product  alternatives  to  determine  if  the 
R&D  is  justified? 

2.  Are  product  implementation  cost  estimates  needed  to  make  decisions  about  how 
the  R&D  should  be  conducted? 

3.  Has  AFHRL  agreed  to  conduct  product  value  estimation  studies  before  the  products 
are  transitioned  to  users  and/or  intermediate  transitioning  agents? 

4.  Have  the  users  identified  the  kind  of  information  that  would  help  them  to  decide 
how  the  products  should  be  implemented? 

5.  Can  the  value  of  the  R&D  products  be  meaningfully  quantified  without  an  excessive 
amount  of  effort? 


F.  Benefit  estimation  FEAs 


2  Most  product  value  estimation  FEAs  are  the  responsibility  of  the  transitioning  agent  (e  g.  the  6  4  agency  responsible  for 
transition)  rather  than  AFHRL.  AFHRL  resources  should  not  be  used  for  product  value  estimation  FEAs  under  these 
circumstances. 
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1  Can  alternatives  with  equal  costs  and  differing  benefits  be  clearly  defined? 

2  Can  the  important  tangible  and  intangible  benefits  be  estimated  in  a  reasonable 
way? 

3  Will  the  information  about  benefits  be  used  to  make  important  decisions  about  the 
conduct  or  utilization  of  the  R&D? 


G  Cost  analysis  FEAs 

1.  Can  alternatives  with  equal  benefits  and  differing  costs  be  clearly  defined? 

2.  Are  future  conditions  understood  well  enough  for  life-cycle  costs  to  be  estimated 
in  a  realistic  fashion? 

3  Is  information  about  the  life-cycle  costs  of  alternative  configurations  needed  to  make 
critical  decisions  about  next  steps? 

4  Has  AFHRL  agreed  to  provide  the  transitioning  agent  (e  g.  a  6.4  agency)  with 
information  about  the  life-cycle  costs  of  alternative  deployment  optic 


H  Advanced  Planning  FEAs 

1  Has  a  basic  plan  for  the  most  logical  R&D  path  been  prepared? 

2  Have  product  values,  product  delivery  dates  and  product  utilization  periods  been 
estimated? 

3  Do  the  advantages  and  disadvantages  of  complex  scheduling  alternatives  need  to 
be  explored? 

4  is  this  the  right  time  to  explore  scheduling  alternatives? 

5  Does  the  R&D  effort  involve  significant  risks? 

6  Is  the  information  about  risks  reliable  enough  to  justify  risk  analysis  studies  at  this 
time? 

7  Could  the  value  of  the  R&O  be  significantly  affected  by  fluctuations  in  key  variables 
or  circumstances? 

8  Is  this  a  good  time  to  conduct  sensitivity  analysis  studies,  or  should  they  be  deferred 
until  more  information  is  available? 

9  Has  the  transitioning  agent  (e  g.  a  6.4  agency)  asked  AFHRL  to  analyze  the  impact 
of  critical  issues  that  influence  product  utilization? 
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D.  Timing  of  FEAs 

Many  different  FEAs  are  possible  before,  during  and  after  an  R&D  project  is  initiated.  It 
may  be  "too  soon"  or  "too  late"  for  one  type  of  FEA,  but  just  the  right  time  to  conduct  an 
FEA  of  another  type. 

Most  exploratory  R&D  projects  at  AFHRL  are  developing  something  that  is  not  clearly 
established,  rather  than  a  firmly  defined  product.  As  a  result,  cost  effectiveness  studies  must 
be  timed  very  carefully.  There  is  not  much  to  be  gained  by  making  guesses  about  where  you 
are  going  and  then  doing  a  lengthy  cost-benefit  analysis  of  the  products  that  logically  follow 
from  those  guesses.  If  you  don’t  have  a  clear  concept  of  what  the  R&D  products  will  look 
like,  it  is  "too  soon"  to  conduct  detailed  cost  effectiveness  studies. 

On  the  other  hand,  if  the  nature  of  the  products  is  already  predetermined  by  Tri-Service 
agreements  or  general  officer  commitments,  there  is  not  much  to  be  gained  by  looking  at 
alternatives  that  have  already  been  excluded.  It  is  usually  "too  late"  to  start  studying  alternative 
configurations  under  such  circumstances. 

Similar  too-soon  versus  too-late  concerns  exist  for  forecasts  of  future  requirements  It  is 
"too  soon"  for  FEAs  of  this  type  if  the  follow-on  work  cannot  be  initiated  for  several  years 
regardless  of  your  findings  (for  example,  if  the  in-house  expertise  to  conduct  the  R&D  were 
not  available).  However,  it  is  "too  late"  if  the  future  requirements  to  be  studied  have  already 
been  established  by  circumstances  beyond  the  Laboratory's  control  (e  g.  a  Major  Command 
or  Tri-Service  agreement  or  a  Congressional  mandate). 


III.  CONTENTS  OF  EAs  AND  FEAs 


A.  Guidance  contained  in  AFR  173*15 


AFR  173-15  contains  useful  information  about  the  suggested  contents  of  EAs,  and  should 
be  examined  very  closely  if  cost  effectiveness  comparisons  are  planned  as  part  of  your  FEA. 
Exhibit  111-1  (abstracted  from  AFR  173-15)  contains  Information  about  the  contents  of  EA  reports 
Note  that  the  content  areas  listed  are  required  rather  than  optional.  Since  AFR  173-15  could 
conceivably  be  revised  after  the  present  guidelines  are  published,  those  who  conduct  EAs  are 
encouraged  to  check  a  current  copy  of  this  regulation  for  guidance 


B.  General  Guidance  Regarding  the  Contents  of  FEAs 

There  is  no  single  model  for  an  FEA,  and  each  FEA  should  reflect  the  nature  of  the  decision 
to  be  made.  General  recommendations  for  the  contents  of  FEA  reports  are  provided  in  Exhibit 
111-2.  Significant  tailoring  of  these  optional  section  headings  is  required  for  each  of  the  eight 
types  of  FEAs  listed  in  Exhibit  11-1. 
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Exhibit  111-1.  Required  Contents  of  EA  Reports 
(from  p.  6  of  AFR  173-15) 


2-1.  General  Information,  Economic  analysis  (or 
cosi-benelil  analvsixi  is  a  method  lor  systematically 
comparing  competing  protect  alternatives  Cost  is 
not  the  sole  eriterion  on  which  project  selection 
should  he  based,  economic  analysis  offers  a  means 
ol  swemicallv  assessing  both  monetary  and  non- 
monetarv  costs  and  benefits  across  alternatives.  An 
economic  analvsis  is  only  as  good  as  the  process  by 
which  it  is  undertaken  Identification  and  valuation 
of  costs  and  benefits  structure  the  whole  analysis, 
therefore,  it  is  essential  that  these  processes  he  con¬ 
ducted  according  to  sound  methodology  and  good 
common  sense  The  guidelines  below  are  designed 
with  this  in  nnnd  Each  economic  analysis  must 
include  at  least  the  following: 

a  An  executive  summary  with  recommendations. 
The  summary  should  be  self-contained  but  concise: 
the  reader  should  be  able  to  grasp  the  basic  facts 
quickly  and  understand  the  recommendation. 

b  A  clear  statemcnf  of  the  problem  or  objective 
ti  e  .  mission  or  mission  support  requirement)  to  be 
met  by  the  alternatives  under  study. 

c  Relevant  assumptions,  criteria,  and  variables 
which  influence  cost  and  effectiveness,  such  as 
required  operational  readiness  dates,  assumptions 
about  future  energy  prices,  etc. 

d  A  complete  list  of  alternatives  considered  to 
meet  the  objective  If  alternatives  to  current  pro¬ 
grams  are  covered  in  the  analysis,  then  the  status 
quo  must  be  explicitly  included  as  a  separate  alterna¬ 
tive 

e  A  thorough  description  of  each  feasible  alterna¬ 
tive  that  could  fulfill  the  program  or  protect  objec¬ 
tive  The  description  of  alternatives  should  include  a 
concise  explanation  of  how  each  process  or  proce¬ 
dure  would  work,  what  personnel,  equ  'mcnt.  or 
facilities  would  be  required,  and  what  other  changes 
would  be  involved  Any  alternative  judged  in.easible 
must  be  identif  ied  and  the  grounds  for  its  rejection 
documented 

f  Estimation  of  costs  and  benefits  of  each  alterna¬ 
tive  The  costs  and  benefits  of  the  altemat.ves  should 
be  summarized;  cost  and  benefit  tables,  ssiurces.  and 
computations  should  be  included  as  attachments.  All 
resources  required  to  achieve  stated  objectives  are  to 
be  shown  With  a  few  exceptions  (analyses  of  lease 
versus  buy  and  other  purely  financial  decisions),  all 
calculations  should  be  in  constant  (real  or  deflated) 
dollars  To  compare  programs  or  protects  more  accu- 
ratelv .  costs  and  benefits  for  each  alternative  should 


be  divided  according  to  whether  they  are  monetary 
in  nature  or  not  Paragraph  2-4  provides  further 
guidelines  for  estimating  costs  and  benefits. 

1 1 )  Monetary  costs  or  benefits  are  those  which 
take  the  form  ol  specific  financial  outlays  or 
receipts  The  list  of  costs  for  each  alternative  should 
be  exhaustive,  but  care  must  be  exercised  to  ensure 
against  double  counting  Specific  treatment  of 
various  cost  elements  is  further  detailed  in  paragraph 
2-4  Monetary  benefits  (such  as  the  proceeds  from 
the  sale  of  assets,  lease  fees,  etc.)  should  be  thor¬ 
oughly  documented.  A  final  calculation  of  dis¬ 
counted  net  costs  (i.c  .  monetary  costs  minus 
monetary  benefits)  should  be  presented  for  each 
alternative. 

|2)  Nonmonetary  costs  and  benefits  are  those 
which  cannot  be  stated  in  doila-  .c’..  A  nonmone- 
tarv  cost  is  a  reduction  of  apt  ility  or  performance 
brought  about  by  the  ..icction  of  a  particular  alterna¬ 
tive:  similarly,  a  nonmonetary  benefit  is  an  enhance¬ 
ment  of  capability  or  performance.  Those 
nonmonetary  *osts  and  benefits  which  lend  them¬ 
selves  to  direct  quantitative  measurement  should  be 
compared  on  that  basis.  Degradations  to  or  enhance¬ 
ments  of  other  programs  should  be  included  as  non¬ 
monetary  costs  or  benefits. 

(a)  Cost  and  benefit  calculations  should  be 
based  on  the  most  accurate  data  available:  case  spe¬ 
cific  data  (i.e..  data  pertaining  to  the  project  or  cir¬ 
cumstances  at  hand)  should  be  used  to  the  greatest 
extent  possible.  Otherwise,  average  values  from  a 
variety  of  sources  can  be  used. 

(b)  When  future  costs  or  benefits  are  uncer¬ 
tain  (e  g.,  because  of  the  nature  of  the  forecasting 
process),  sensitivity  analysis  should  be  used  to  eval¬ 
uate  the  risk  attending  the  estimate  used 

g  A  summarization  of  each  alternative.  A  com¬ 
parison  should  be  made  showing  the  relative 
strengths  and  weaknesses  of  each  alternative  and 
identifying  the  most  effective  alternative  for  accom¬ 
plishing  the  mission  objective  This  comparison 
should  include  all  life-cycle  dollar  costs  and  benefits 
and  nonmonetary  costs  and  benefits  not  common  to 
all  of  the  alternatives. 
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Exhibit  III-2.  Suggested  Contents  Of  FEA  Reports 


Summary 

User  Requirement 
Base  Case 

Relevant  Technology 

/stems  Change 
Alternatives 

Plans  for  R&D 

Scope  of  FEA 

Options  Considered 

Costs 

Benefits 


Effectiveness 

Comparisons 


Conclusions 


An  executive  summary  of  the  information  contained  in  the  report, 
including  recommendations  and  the  analyst’s  overall  evaluation 

A  description  of  the  problem  or  objective  as  it  is  seen  by  the  user. 

A  description  of  the  operational  systems  as  they  exist  now,  as  well 
as  the  changes  that  are  expected  to  occur  independently  of  the 
proposed  R&D  as  a  natural  result  of  external  events  and  management 
decisions. 

A  description  of  the  current  state  of  the  art  in  the  technology  that 
would  be  used  to  meet  user  requirements 

A  description  of  systems  change  alternatives,  including  low  cost 
alternatives  to  the  R&D  products  that  would  be  developed  by  the 
proposed  R&D  as  well  as  the  alternative  of  letting  the  system  evolve 
without  R&D  by  AFHRL. 

This  should  include:  the  objectives  of  the  proposed  R&D,  the  products 
to  be  developed,  the  technical  methods  used  to  develop  the  R&D 
products,  and  a  description  of  parallel  R&D  efforts  with  which  the 
proposed  R&D  must  be  coordinated. 

This  should  include  the  specific  objectives  of  the  FEA,  the  assumptions 
used  to  estimate  costs  and  benefits,  and  a  description  of  how  the 
FEA  data  were  obtained. 

The  who,  what,  where,  when,  and  why  of  the  options  that  were 
considered  as  part  of  the  analysis. 

R&D  costs,  deployment  and  O&S  costs,  cost  avoidance  expectations, 
total  life-cycle  costs  of  alternative  configurations. 

The  term  "benefits'1  does  not  include  cost  avoidance  values  (e  g 
training  cost  avoidance,  accident  cost  avoidance),  which  are  treated 
separately  as  cost  adjustment  factors.  Examples  of  benefits  are: 
increased  sortie  rate,  greater  availability  of  in-flight  hours  to  practice 
critical  tasks,  more  sustainable  operations  at  dispersed  bases,  and 
enhanced  combat  performance. 

Cost  effectiveness  comparisons  of  significant  options,  usually  including 
the  baseline  condition  and  one  or  more  systems  change  alternatives 
Narrative  descriptions  of  benefits  and  rank  orderings  of  alternatives 
are  permitted  when  quantification  is  not  feasible  or  practical. 

This  section  describes  the  implications  of  the  analysis  for  the  proposed 
R&D  and  the  basis  for  the  analyst’s  conclusions  about  it. 
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C.  Sources  of  Information  about  Costs 


AFR  173-13,  US  Air  Force  Cost  and  Planning  Factors,  provides  cost  information  and  ground 
rules  that  can  be  used  to  estimate  resource  requirements  and  costs  associated  with  Air  Force 
structures,  missions,  and  activities.  Chapter  headings  in  the  197-page  document  are:  General 
Information,  Logistics  Factors,  Personnel  Factors,  Programming  Factors,  Inflation  Factors,  Attrition 
Factors,  and  Cost  Models. 

Another  useful  source  of  information  is  the  Air  Force  Almanac,  which  is  published  in  the 
May  issue  of  Air  Force  Magazine.  The  information  contained  in  this  unofficial  source  (which 
includes  estimated  purchases  of  new  weapon  systems)  has  an  advantage  in  that  the  analyst 
need  not  be  concerned  about  the  possibility  that  the  information  used  in  the  analysis  might  be 
classified  or  procurement-sensitive  in  some  way. 

Current  information  about  the  Office  of  the  Secretary  of  Defense  inflation  rates  and  a  variety 
of  cost  models  can  be  obtained  from  the  Air  Force  Cost  Center.  The  Air  Training  Command 
(ATC)  Cost  Handbook,  which  is  maintained  by  ATC  headquarters  and  revised  annually,  is  a 
useful  source  of  information  about  training  costs. 


D.  Specific  Content  Recommendations 


Specific  content  recommendations  for  each  of  the  eight  types  of  reports  are  contained  in 
exhibit  111-3. 
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Exhibit  111-3.  Suggested  Contents  of  FEA  Reports 


FEAs  develop  and  present  Information  that  helps  us  make  better  decisions  about  new  products  AFHRL  should 
develop  for  the  Air  Force  and  ways  in  which  the  Laboratory  should  develop  them  Suggested  ronte"t«  o*  PEA 
reports  are: 

a.  Forecasts  of  Future  Requirements.  These  forecasts  provide  systematic  information  about  the  future  environments 
in  which  the  Air  Force  will  have  to  carry  out  its  missions.  They  also  identify  the  capabilities  required  to  carry  out 
the  missions  successfully.  Forecasts  of  future  requirements  should  include  descriptions  of  the  baseline  situation, 
forecasted  changes  in  enemy  capabilities,  combat  implications,  Air  Force  problems,  available  solutions  to  the  problems, 
and  solutions  requiring  R&D. 

b.  Assessments  of  Emerging  Technologies  These  assessments  provide  information  about  the  potentials  of 
emerging  technologies  for  enhancing  mission  effectiveness  and  their  possible  consequences  for  future  requirements 
They  should  include  descriptions  of  the  current  state  of  the  art,  of  ongoing  and  planned  R&D,  and  of  specific  Air 
Force  problems  that  could  be  alleviated  or  solved  by  the  new  technology. 

c.  Definitions  and  Evaluations  of  Alternative  System  Configurations  and  Deployment  Strategies  Results  provide 
information  about  the  likely  costs  and  effectiveness  of  conceptually  different  systems  that  could  be  developed  They 
provide  information  for  deciding  what  new  systems  to  develop.  These  FEAs  should  include  projections  of  the  baseline 
situation,  systems  definitions,  alternative  systems  configurations  and  deployment  strategies,  estimates  ot  lifecycle 
costs,  estimates  of  the  effectiveness  associated  with  each  system  alternative  or  deployment  strategy,  and  an  assessmeni 
of  the  likely  efficiency  of  each  system  concept  in  carrying  out  the  Air  Force  mission 

d.  R&D  Plans  Evaluations.  Plans  evaluations  provide  information  useful  in  deciding  how  to  achieve  an  R&D 
goal.  They  should  include  descriptions  of  the  R&D  objectives,  time  and  cost  constraints,  important  milestones  and 
coordination  requirements,  joint  effort  possibilities,  network  planning  diagrams  for  alternative  R&D  strategies,  and 
estimates  of  the  cost  and  time  required  to  carry  out  each  R&D  strategy. 

e.  Product  Value  Estimation  Studies.  These  studies  provide  information  useful  for  deciding  whether  to  develop 
a  product.  They  provide  Information  about  the  likely  costs  and  benefits  of  R&D  products,  if  the  products  were  to 
be  implemented.  Product  value  estimation  studies  should  include  descriptions  of  the  R&D  products,  alternative 
subsystem  configurations  and  utilization  plans,  assertions  underlying  cost  and  benefit  estimates,  life-cycle  cost 
estimates,  benefit  estimates,  and  cost-benefit  comparisons  for  alternative  subsystems  and  utilization  plans 

f.  Benefit  Estimation  Studies.  Such  studies  provide  information  about  the  nature  and  likely  magnitude  of  the 
benefits  that  would  stem  from  the  operational  use  of  a  new  product.  They  should  include  functional  descriptions 
of  how  a  new  product  would  benefit  the  Air  Force,  descriptions  of  benefit-estimation  and  data-collection  methodologies, 
documentation  of  the  relationships  between  system  characteristics,  deployment  strategies  and  benefit  levels,  and 
comparisons  of  benefit  levels.  Benefit  estimation  studies  can  support  product  value  estimation  and  planning  studies. 

g.  Life-Cycle  Cost  Analyses.  These  analyses  provide  information  regarding  how  much  it  would  cost  the  Air 
Force  to  develop,  implement,  and  use  an  R&D  product  or  system.  They  should  include  descriptions  ot  the  alternatives 
to  be  evaluated,  enumeration  of  all  cost  elements,  documentation  of  all  cost  estimating  parameters/relationships/modeis, 
and  the  life-cycle  costs  estimated.  Air  Force  policy  regarding  the  computation  of  life-cycle  costs  (AFR  800-11)  should 
be  followed.  Life-cycle  cost  analyses  can  support  evaluations  of  alternative  system  configurations  and  deployment 
strategies,  product  value  estimation  studies,  and  planning  studies. 

h.  Planning  Studies.  These  are  studies  that  require  the  use  of  network  diagram  and  value  estimation  techniques 
which  help  answer  questions  about  the  ways  in  which  existing  R&D  plans  might  be  changed.  Examples  ot  planning 
studies  are:  Schedule  analysis,  risk  analysis,  and  sensitivity  analysis.  For  schedule  analysis  studies,  network  diagrams 
are  prepared  in  which  separate  time,  resource  requirement,  and  benefit  estimates  are  obtained  for  two  or  more 
scheduling  options.  The  impact  of  the  alternative  schedules  on  benefit  periods,  costs,  and  benefits  is  then  determined 
Schedule  analysis  studies  are  done  when  the  length  of  the  benefit  period  is  a  critical  factor  or  when  a  "crash" 
schedule  has  been  proposed.  Risk  analysis  studies  evaluate  the  risks  involved  in  conducting  an  R&D  effort,  with 
emphasis  on  the  cost/benefit  consequences  of  alternative  events  and  research  and  development  outcomes  Risk 
analysis  studies  are  appropriate  when  reasonable  estimates  can  be  made  of  the  probabilities  that  the  alternative 
events  that  are  expected  to  affect  R&D  outcomes  will  occur.  Sensitivity  analysis  studies  examine  the  effect  obtained 
by  repetitive  changes  in  the  direction  and/or  magnitude  of  the  cost/benefit  values  embedded  in  the  analysis  Sensitivity 
analysis  studies  are  appropriate  when  events  are  uncertain  and  Information  is  needed  about  the  impact  that  alternative 
cost  and  benefit  values  might  have  on  the  overall  value  of  an  effort. 
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IV.  DOCUMENTING  FEAs 


A.  R&D  Case  File  Documentation 

FEA  information,  like  all  important  forms  of  information  about  the  conduct  of  an  R&D  effort, 
is  maintained  in  the  R&D  case  file.  AFR  12-50  (Vol.  II),  Table  80-2,  contains  guidance  regarding 
the  content  of  R&D  case  files. 


B.  Documentation  Maintained  by  AFHRL/SA 


As  required  by  AFHRLR  173-1,  the  Special  Projects  Office  (AFHRL/SA)  makes  recommendations 
regarding  FEAs  following  the  annual  “program  call"  in  which  work  unit  validation  decisions  by 
Division  Chiefs  are  reviewed  by  the  AFHRL  Commander  and  his  staff.  SA  analysts  also  track 
FEAs  for  6  3  projects  during  staff  assistance  visits. 

Information  about  the  need  for  FEAs  is  documented  in  the  Commander’s  "program  call" 
evaluation  letters.  Information  about  division  progress  in  conducting  FEAs  is  documented  in 
SA  staff  assistance  visit  reports. 


V.  GENERAL  RECOMMENDATIONS  FOR  Ft  As 

The  following  "Do’s"  and  "Don't’s"  are  offered  for  your  consideration  when  FEAs  are  being 
planned,  conducted,  and  reported. 


OBJECTIVES 

DO... 

DON’T ... 

Conduct  a  rational  and  independent 
study  of  the  advantages  and  disad¬ 
vantages  of  the  alternatives  under  con¬ 
sideration. 

Permit  the  design  of  a  study  to  be 
influenced  by  pressures  that  may 
exist  for  or  against  a  proposal. 

Study  alternatives  that  have  a  high 
probability  of  occurring. 

Study  alternatives  that  have  a  low 
probability  of  occurring. 

Give  priority  in  study  selection 
decisions  to  those  items  that  have  the 
most  important  source  implications 
and  the  greatest  payoff  potential. 

Give  priority  to  studies  that  won’t 
have  much  impact  when  they  are 
finished. 

STUDY  FOCUS 

DO... 

DON'T... 

Give  first  priority  to  your  own  under¬ 
standing  of  the  problem. 

Assume  that  your  initial  views  of  what 
is  needed  are  correct. 

Give  second  priority  to  base  case 
definition  if  the  base  case  is  unclear. 

Start  work  without  fully  understanding 
the  baseline  conditions. 

Give  third  priority  to  your  thoughts 
about  systems  change  alternatives. 

Accept  existing  systems  alternatives 
without  question. 
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Focus  upon  costs  and  benefits  that  Try  to  include  all  costs  and  benefits 
make  a  difference.  that  could  be  considered  as  part  of 

every  analysis. 

Screen  out  alternatives  that  are  not  Try  to  analyze  all  of  the  alternatives 
competitive  when  analytic  com-  that  could  be  analyzed 
parisons  are  made. 

STUDY  DO...  DON’T 

DESIGN 

Design  each  analysis  as  a  completed  Defer  conclusions  until  additional 
study.  analyses  have  been  conducted 

Design  FEAs  to  facilitate  Important  Design  FEAs  to  focus  upon  the  less 
research  decisions  and  resource  al-  important  decisions 
location  decisions. 

Focus  on  future  conditions  that  will  Spend  too  much  time  on  existing 
be  impacted  by  the  R&D  products  to  conditions 
be  developed. 

SCOPE  OF  DO...  DON'T.  . 

ANALYSIS 

Consider  the  possibility  that  the  re-  Assume  that  existing  requirements 

qulrement  or  the  work  schedule  could  and  work  schedules  should  not  be 

be  modified  in  ways  that  would  im-  changed  in  any  way. 

prove  the  value  of  the  R&D  to  the 
Air  Force. 

Focus  the  analysis  upon  those  costs  Try  to  include  all  of  the  costs  and 

and  benefits  that  are  most  relevant  benefits  that  could  possibly  be  con- 

for  the  options  under  consideration.  sidered 

METHODS  DO...  DON'T 

List  the  potential  benefits  and  describe  List  benefits  without  explaining  their 
the  scientific  and  technical  bases  upon  rationale, 
which  the  benefits  are  projected. 

Use  ratings  and  rankings  when  hard  Try  to  justify  with  quantitative  data 
data  are  not  available  every  point  that  is  made. 

Ask  personnel  outside  of  your  or-  Ask  personnel  outside  of  your  or¬ 
ganization  for  information  that  is  either  ganization  for  information  that  would 

readily  available  or  can  be  quickly  be  difficult  and  time  consuming  to 

estimated.  provide. 

Try  to  develop  specific  estimates  of  Accept  very  general  statements  like 
costs  and  benefits.  "improve  readiness." 
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REPORTS 


Use  shortcut  quantification  methods 
when  necessary,  such  as  assumption- 
based  estimates,  percentage  estimates 
of  possible  values,  and  illustrative 
values. 

Use  order  of  magnitude  (one  or  two 
significant  digits)  value  estimates  when 
necessary. 


Make  preliminary  assessments  of  the 
sensitivity  of  results  to  possible 
changes  in  values. 

Analyze  risks  that  have  a  good  chance 
of  impacting  the  success  of  the 
proposed  R&D. 

Consider  the  impact  of  important 
scheduling  options  on  benefit  periods. 

Try  to  modify  readily  available  data 
for  your  use. 

DO 

Identify  unresolved  questions  in  you r 
report. 


Briefly  indicate  any  uncertainties  in 
the  data  and  the  sensitivity  of  the 
results  to  the  analytic  approach  that 
was  used 


Insist  upon  hard  data  when  the  situa¬ 
tion  and/or  the  time  constraints  don't 
permit  it. 


Give  the  appearance  of  having  num¬ 
bers  with  multi-digit  accuracy  when 
that  degree  of  accuracy  does  not 
really  exist. 

Conduct  complex  sensitivity  analysis 
studies  that  are  not  really  necessary. 


Try  to  analyze  every  risk  involved  in 
an  R&D  effort. 


Let  the  accidents  of  budget  determine 
the  duration  of  the  b*'  .  leriods. 

Develop  sor>*  .sticated  projection 
methods  ,h3t  are  not  really  needed 

DON’T... 

Violate  the  constraint  that  each  report 
must  stand  on  its  own  merits  as  a 
completed  study. 

Give  the  impression  that  the  data  are 
better  than  they  really  are. 
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